The present invention relates to an image sensor, and more particularly but not exclusively to two and three-dimensional optical processing from within restricted spaces, and an endoscope using the same.
Endoscopy is a surgical technique that involves the use of an endoscope, to see images of the body""s internal structures through very small incisions.
Endoscopic surgery has been used for decades in a number of different procedures, including gall bladder removal, tubal ligation, and knee surgery, and recently in plastic surgery including both cosmetic and re-constructive procedures.
An endoscope may be a rigid or flexible endoscope which consists of five basic parts: a tubular probe, a small camera head, a camera control unit, a bright light source and a cable set which may include a fiber optic cable. The endoscope is inserted through a small incision; and connected to a viewing screen which magnifies the transmitted images of the body""s internal structures.
During surgery, the surgeon is able to view the surgical area by watching the screen while moving the tube of the endoscope through the surgical area.
In a typical surgical procedure using an endoscope, only a few small incisions, each less than one inch long, are needed to insert the endoscope probe and other instruments. For some procedures, such as breast augmentation, only two incisions may be necessary. For others, such as a forehead lift, three or four short incisions may be needed. The tiny eye of the endoscope camera allows a surgeon to view the surgical site.
An advantage of the shorter incisions possible when using an endoscope is reduced damage to the patient""s body from the surgery. In particular, the risk of sensory loss from nerve damage is decreased. However, most current endoscopes provide only flat, two-dimensional images which are not always sufficient for the requirements of the surgery. The ability of an endoscope to provide three-dimensional information in its output would extend the field of endoscope use within surgery.
The need for a 3D imaging ability within an endoscope has been addressed in the past. A number of solutions that provide stereoscopic images by using two different optical paths are disclosed in U.S. Pat. No. 5,944,655, U.S. 5,222,477, U.S. 4,651,201, U.S. 5,191,203, U.S. 5,122,650, U.S. 5,471,237, JP7163517A, U.S. 5,673,147, U.S. 6,139,490, U.S. 5,603,687, WO9960916A2, and JP63244011A.
Another method, represented by US Patents, U.S. Pat. No. 5,728,044 and U.S. 5,575,754 makes use of an additional sensor that provides location measurements of image points. Patent JP8220448A discloses a stereoscopic adapter for a one-eye endoscope, which uses an optical assembly to divide and deflect the image to two sensors. A further method, disclosed in U.S. Pat. No. 6,009,189 uses image acquisition from different directions using one or more cameras. An attempt to obtain 3D information using two light sources was disclosed in U.S. Pat. No. 4,714,319 in which two light sources are used to give an illusion of a stereoscopic image based upon shadows. JP131622A discloses a method for achieving the illusion of a stereoscopic image by using two light sources, which are turned on alternately.
An additional problem with current endoscopes is the issue of lighting of the subject for imaging. The interior spaces of the body have to be illuminated in order to be imaged and thus the endoscope generally includes an illumination source. Different parts of the field to be illuminated are at different distances from the illumination source and relative reflection ratios depend strongly on relative distances to the illumination source. The relative distances however may be very large In a typical surgical field of view, distances can easily range between 2 and 20 cm giving a distance ratio of 1:10. The corresponding brightness ratio may then be 1:100, causing blinding and making the more distant object all but invisible.
One reference, JP61018915A, suggests solving the problem of uneven lighting by using a liquid-crystal shutter element to reduce the transmitted light. Other citations that discuss general regulation of illumination levels include U.S. Pat. No. 4,967,269, JP4236934A, JP8114755A and JP8024219A.
In general it is desirable to reduce endoscope size and at the same time to improve image quality. Furthermore, it is desirable to produce a disposable endoscope, thus avoiding any need for sterilization, it being appreciated that sterilization of a complex electronic item such as an endoscope being awkward in itself.
Efforts to design new head architecture have mainly concentrated on integration of the sensor, typically a CCD based sensor, with optics at the distal end. Examples of such integration are disclosed in U.S. Pat. No. 4,604,992, U.S. 4,491,865, U.S. 4,692,608, JP60258515A, U.S. Pat. No. 4,746,203, U.S. Pat. No. 4,720,178, U.S. 5,166,787, U.S. 4,803,562, U.S. 5,594,497 and EP434793B1. Reducing the overall dimensions of the distal end of the endoscope are addressed in U.S. Pat. No. 5,376,960 and No. 4,819,065, and Japanese Patent Applications No. 7318815A and No. 70221A. Integration of the endoscope with other forms of imaging such as ultrasound and Optical Coherence Tomography are disclosed in U.S. Pat. No. 4,869,256, U.S. 6,129,672, U.S. 6,099,475, U.S. 6,039,693, U.S. 55,022,399, U.S. 6,134,003 and U.S. 6,010,449.
Intra-vascular applications are disclosed in certain of the above-mentioned patents, which integrate the endoscope with an ultrasound sensor or other data acquisition devices. Patents that disclose methods for enabling visibility within opaque fluids are U.S. Pat. No. 4,576,146, U.S. 4,827,907, U.S. 5,010,875, U.S. 4,934,339, U.S. 6,178,346 and U.S. 4,998,972.
Sterilization issues of different devices including endoscopes are discussed in WO9732534A1, U.S. Pat. No. 5,792,045 and U.S. 5,498,230. In particular JP3264043A discloses a sleeve that was developed in order to overcome the need to sterilize the endoscope.
The above-mentioned solutions are however incomplete and are difficult to integrate into a single endoscope optimized for all the above issues.
It is an aim of the present embodiments to provide solutions to the above issues that can be integrated into a single endoscope.
It is an aim of the embodiments to provide an endoscope that is smaller than current endoscopes but without any corresponding reduction in optical processing ability.
It is a further aim of the present embodiments to provide a 3D imaging facility that can be incorporated into a reduced size endoscope.
It is a further aim of the present embodiments to provide object illumination that is not subject to high contrast problems, for example by individual controlling of the light sources.
It is a further aim of the present embodiments to provide a modified endoscope that is simple and cost effective to manufacture and may therefore be treated as a disposable item.
Embodiments of the present invention provide 3D imaging of an object based upon photometry measurements of reflected light intensity. Such a method is relatively efficient and accurate and can be implemented within the restricted dimensions of an endoscope.
According to a first aspect of the present invention there is thus provided a pixilated image sensor for insertion within a restricted space, the sensor comprising a plurality of pixels arranged in a selected image distortion pattern, said image distortion pattern being selected to project an image larger than said restricted space to within said restricted space substantially with retention of an image resolution level.
Preferably, the image distortion pattern is a splitting of said image into two parts and wherein said pixilated image sensor comprises said pixels arranged in two discontinuous parts.
Preferably, the discontinuous parts are arranged in successive lengths.
Preferably, the restricted space is an interior longitudinal wall of an endoscope and wherein said discontinuous parts are arranged on successive lengths of said interior longitudinal wall.
Preferably, the restricted space is an interior longitudinal wall of an endoscope and wherein said discontinuous parts are arranged on successive lengths of said interior longitudinal wall.
Preferably, the distortion pattern is an astigmatic image distortion.
Preferably, the distortion pattern is a projection of an image into a rectangular shape having dimensions predetermined to fit within said restricted space.
A preferred embodiment includes one of a group comprising CMOS-based pixel sensors and CCD based pixel sensors.
A preferred embodiment is controllable to co-operate with alternating image illumination sources to produce uniform illuminated images for each illumination source.
According to a second aspect of the present invention there is provided an endoscope having restricted dimensions and comprising at least one image gatherer, at least one image distorter and at least one image sensor shaped to fit within said restricted dimensions, and wherein said image distorter is operable to distort an image received from said image gatherer so that the image is sensible at said shaped image sensor substantially with an original image resolution level.
Preferably, the image distorter comprises an image splitter operable to split said image into two part images.
Preferably, the image sensor comprises two sensor parts, each separately arranged along longitudinal walls of said endoscope.
Preferably, the two parts are arranged in successive lengths along opposite longitudinal walls of said endoscope.
Preferably, the distorter is an astigmatic image distorter.
Preferably, the astigmatic image distorter is an image rectangulator and said image sensor comprises sensing pixels rearranged to complement rectangulation of said image by said image rectangulator.
Preferably, the image distorter comprises at least one lens.
Preferably, the image distorter comprises at least one image-distorting mirror.
Preferably, the image distorter comprises optical fibers to guide image light substantially from said lens to said image sensor.
Preferably, the image distorter comprises a second lens.
Preferably, the image distorter comprises at least a second image-distorting mirror.
Preferably, the image distorter comprises at least one flat optical plate.
A preferred embodiment comprises at least one light source for illuminating an object, said light source being controllable to flash at predetermined times.
A preferred embodiment comprises a second light source, said first and said second light sources each separately controllable to flash.
Preferably, the first light source is a white light source and said second light source is an IR source.
In a preferred embodiment, one light source being a right side light source for illuminating an object from a first side and the other light source being a left side light source for illuminating said object from a second side.
In a preferred embodiment, one light source comprising light of a first spectral response and the other light source comprising light of a second spectral response.
A preferred embodiment further comprises color filters associated with said light gatherer to separate light from said image into right and left images to be fed to respective right and left distance measurers to obtain right and left distance measurements for construction of a three-dimensional image.
In a preferred embodiment, said light sources are configured to flash alternately or simultaneously.
A preferred embodiment further comprises a relative brightness measurer for obtaining relative brightnesses of points of said object using respective right and left illumination sources, thereby to deduce 3 dimensional distance information of said object for use in construction of a 3 dimensional image thereof.
A preferred embodiment further comprises a second image gatherer and a second image sensor.
Preferably, the first and said second image sensors are arranged back to back longitudinally within said endoscope.
Preferably, the first and said second image sensors are arranged successively longitudinally along said endoscope.
Preferably, the first and said second image sensors are arranged along a longitudinal wall of said endoscope.
A preferred embodiment comprises a brightness averager operable to identify brightness differentials due to variations in distances from said endoscope of objects being illuminated, and substantially to cancel said brightness differentials.
A preferred embodiment further comprises at least one illumination source for illuminating an object with controllable width light pulses and wherein said brightness averager is operable to cancel said brightness differentials by controlling said widths.
A preferred embodiment has at least two controllable illumination sources, one illumination source for emitting visible light to produce a visible spectrum image and one illumination source for emitting invisible (i.e. IR or UV) light to produce a corresponding spectral response image, said endoscope being controllable to produce desired ratios of visible and invisible images.
According to a third aspect of the present invention there is provided an endoscope system comprising an endoscope and a controller, said endoscope comprising:
at least one image gatherer,
at least one image distorter and
at least one image sensor shaped to fit within restricted dimensions of said endoscope, said image distorter being operable to distort an image received from said image gatherer so that the image is sensible at said shaped image sensor with retention of image resolution,
said controller comprising a dedicated image processor for processing image output of said endoscope.
Preferably, the dedicated image processor is a motion video processor operable to produce motion video from said image output.
Preferably, the dedicated image processor comprises a 3D modeler for generating a 3D model from said image output.
Preferably, the said dedicated image processor further comprises a 3D imager operable to generate a stereoscopic display from said 3D model.
A preferred embodiment comprises an image recorder for recording imaging.
A preferred embodiment comprises a control and display communication link for remote control and remote viewing of said system.
Preferably, the image distorter comprises an image splitter operable to split said image into two part images.
Preferably, the image sensor comprises two sensor parts, each separately arranged along longitudinal walls of said endoscope.
Preferably, the two parts are arranged in successive lengths along opposite longitudinal walls of said endoscope.
Preferably, the distorter is an astigmatic image distorter.
Preferably, the astigmatic image distorter is an image rectangulator and said image sensor comprises sensing pixels rearranged to complement rectangulation of said image by said image rectangulator.
Preferably, the image distorter comprises at least one lens.
Preferably, the image distorter comprises at least one image-distorting mirror.
Preferably, the image distorter comprises optical fibers to guide image light substantially from said lens to said image sensor.
Preferably, the image distorter comprises a second lens.
Preferably, the image distorter comprises at least a second image-distorting mirror.
Preferably, the image distorter comprises at least one flat optical plate.
A preferred embodiment further comprises at least one light source for illuminating an object.
A preferred embodiment comprises a second light source, said first and said second light sources each separately controllable to flash.
Preferably, the first light source is a white light source and said second light source is an invisible source.
In a preferred embodiment, one light source is a right side light source for illuminating an object from a first side and the other light source is a left side light source for illuminating said object from a second side.
In a preferred embodiment, one light source comprises light of a first spectral response and the other light source comprises light of a second spectral response.
A preferred embodiment comprises color filters associated with said light gatherer to separate light from said image into right and left images to be fed to respective right and left distance measurers to obtain right and left distance measurements for construction of a three-dimensional image.
Preferably, the light sources are configured to flash alternately or simultaneously.
A preferred embodiment further comprises a relative brightness measurer for obtaining relative brightnesses of points of said object using respective right and left illumination sources, thereby to deduce 3 dimensional distance information of said object for use in construction of a 3 dimensional image thereof.
A preferred embodiment further comprises a second image gatherer and a second image sensor.
Preferably, the first and said second image sensors are arranged back to back longitudinally within said endoscope.
Preferably, the first and said second image sensors are arranged successively longitudinally along said endoscope.
Preferably, the first and said second image sensors are arranged along a longitudinal wall of said endoscope.
A preferred embodiment comprises a brightness averager operable to identify brightness differentials due to variations in distances from said endoscope of objects being illuminated, and substantially to reduce said brightness differentials.
According to a fifth embodiment of the present invention there is provided an endoscope for internally producing an image of a field of view, said image occupying an area larger than a cross-sectional area of said endoscope, the endoscope comprising:
an image distorter for distorting light received from said field of view into a compact shape, and
an image sensor arranged in said compact shape to receive said distorted light to form an image thereon.
A preferred embodiment comprises longitudinal walls, wherein said image sensor is arranged along said longitudinal walls, the endoscope further comprising a light diverter for diverting said light towards said image sensor.
Preferably, the image sensor comprises two parts, said distorter comprises an image splitter for splitting said image into two parts, and said light diverter is arranged to send light of each image part to a respective part of said image sensor.
Preferably, the sensor parts are aligned on facing lengths of internal sides of said longitudinal walls of said endoscope.
Preferably, the sensor parts are aligned successively longitudinally along an internal side of one of said walls of said endoscope.
A preferred embodiment of the image distorter comprises an astigmatic lens shaped to distort a square image into a rectangular shape of substantially equivalent area.
A preferred embodiment further comprises a contrast equalizer for compensating for high contrasts differences due to differential distances of objects in said field of view.
A preferred embodiment comprises two illumination sources for illuminating said field of view.
In a preferred embodiment, the illumination sources are controllable to illuminate alternately, and said image sensor is controllable to gather images in synchronization with said illumination sources thereby to obtain independently illuminated images.
In a preferred embodiment, each illumination source is of a different predetermined spectral response.
A preferred embodiment of said image sensor comprises pixels, each pixel being responsive to one of said predetermined spectral responses.
A preferred embodiment of the image sensor comprises a plurality of pixels responsive to white light.
In a preferred embodiment, said image sensor comprises a plurality of pixels responsive to different wavelengths of light.
In a preferred embodiment, the wavelengths used comprise at least three of red light, green light, blue light and infra-red light.
In a preferred embodiment, a second image sensor forms a second image from light obtained from said field of view.
In a preferred embodiment, said second image sensor is placed in back to back relationship with said first image sensor over a longitudinal axis of said endoscope.
In a preferred embodiment, the second image sensor is placed in end to end relationship with said first image sensor along a longitudinal wall of said endoscope.
In a preferred embodiment, the second image sensor is placed across from said first image sensor on facing internal longitudinal walls of said endoscope.
According to a sixth embodiment of the present invention there is provided a compact endoscope for producing 3D images of a field of view, comprising a first image sensor for receiving a view of said field through a first optical path and a second image sensor for receiving a view of said field through a second optical path, and wherein said first and said second image sensors are placed back to back along a longitudinal axis of said endoscope.
According to a seventh embodiment of the present invention there is provided a compact endoscope for producing 3D images of a field of view, comprising a first image sensor for receiving a view of said field through a first optical path and a second image sensor for receiving a view of said field through a second optical path, and wherein said first and said second image sensors are placed end to end along a longitudinal wall of said endoscope.
According to an eighth embodiment of the present invention there is provided a compact endoscope for producing 3D images of a field of view, comprising two illumination sources for illuminating said field of view, an image sensor for receiving a view of said field illuminated via each of said illumination sources, and a view differentiator for differentiating between each view.
Preferably, the differentiator is a sequential control for providing sequential operation of said illumination sources.
Preferably, the illumination sources are each operable to produce illumination at respectively different spectral responses and said differentiator comprises a series of filters at said image sensor for differentially sensing light at said respectively different spectral responses.
Preferably, the image distorter comprises a plurality of optical fibers for guiding parts of a received image to said image sensor according to said distortion pattern.
According to a ninth embodiment of the present invention there is provided a method of manufacturing a compact endoscope, comprising:
providing an illumination source,
providing an image distorter,
providing an image ray diverter,
providing an image sensor whose shape has been altered to correspond to a distortion built into said image distorter, said distortion being selected to reduce at least one dimension of said image sensor to less than that of an undistorted version being sensed,
assembling said image distorter, said image ray diverter and said image sensor to form an optical path within an endoscope.
According to a tenth embodiment of the present invention there is provided a method of obtaining an endoscopic image comprising:
illuminating a field of view,
distorting light reflected from said field of view such as to form a distorted image of said field of view having at least one dimension reduced in comparison to an equivalent dimension of said undistorted image, and
sensing said light within said endoscope using at least one image sensor correspondingly distorted.